MalFe

Please wait ...

Public Datasets

File Operations for Ransomware Detection Rows: 2652 | Cols: 23 | Reports Used: 3084

Description: This dataset explores using various file operations as features.

Dataset SHA256: 6c1b2b7904f20d2691141fcd487763a38fadf273d1f9ff4ba3b4a28e877b1f75

Features (sample):

	sha256	label	PercentageOfFileAPICalls	PercentageOfUniqueFiles	PercentageOfUniqueFileLocations	PercentageOfUniqueFileExtensions	PercentageOfPotentialCustomExtensions	PercentageOfRansomwareExtensions	PercentageOfFileExtensionsDocuments	PercentageOfFileExtensionsImages	PercentageOfFileExtensionsVideos	PercentageOfFileExtensionsAudio	PercentageOfFileExtensionsDatabase	PercentageOfFileExtensionsArchives	PercentageOfFileExtensionsExecutables	PercentageOfFileExtensionsSystem	PercentageOfFileExtensionsBackup	PercentageOfFileExtensionsEmail	PercentageOfFileExtensionsVMImages	PercentageOfFileExtensionsGames	PercentageOfFileExtensionsDevelopment	APICallsPerSecond	SHA256
0	3b4497c7f8c89bf22c984854ac7603573a53b95ed147e80c0f19e549e2b65693	M	60	4	4	1	2	14	2	0	0	0	0	0	96	0	0	0	0	0	0	72	3b4497c7f8c89bf22c984854ac7603573a53b95ed147e80c0f19e549e2b65693
1	8fec485e47fd1231aeb1a4107a4918f92c2b15fa66e9171be39a765d26a12acb	M	12	25	25	25	0	0	0	0	0	0	0	0	100	0	0	0	0	0	0	75	8fec485e47fd1231aeb1a4107a4918f92c2b15fa66e9171be39a765d26a12acb
...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...
2650	7d809e8c9b98c16647bbfac49854c28ecc3fe6d4345410deeaa79445cc50cf51	M	8	14	14	14	100	0	0	0	0	0	0	0	0	0	0	0	0	0	0	16	7d809e8c9b98c16647bbfac49854c28ecc3fe6d4345410deeaa79445cc50cf51
2651	7d6d38f2cbe320aff29eb02998476e731d02ca27ca0e2f79063b207fc10229e8	M	97	0	0	0	0	1	0	0	0	0	0	91	0	0	0	0	0	0	9	6892	7d6d38f2cbe320aff29eb02998476e731d02ca27ca0e2f79063b207fc10229e8

2652 rows × 23 columns

Created by: Avinash
Downloaded 1 times
Comments Enabled: True
Comments: 0

def process(data, filename, sha256, category, malicious):
    try:
        # Implement your data processing here, this function will get called with the data of each 
        # report in the category you have selected along with the filename of the report and hash. 
        # Do your processing and add your data to the features array, and return the features_lables, features, and sha256
        # PLEASE NOTE: THE CODE BELOW SERVES AS AN EXAMPLE ONLY! This example extracts the entropy of the PE Sections. Which can be seen here: https://malfe.cs.up.ac.za/datasets/view/Entropy_of_PE_Sections/5
        ###

analysis_dictionary = {}
        analysis_dictionary["stats"] = {}
        analysis_dictionary["summary"] = {}

if "behavior" in data:

if "summary" in data["behavior"]:

if "file_deleted" in data["behavior"]["summary"]:
                    analysis_dictionary["summary"]["file_deleted"] = {}
                    analysis_dictionary["summary"]["file_deleted"]["data"] = data["behavior"]["summary"]["file_deleted"]
                    analysis_dictionary["summary"]["file_deleted"]["total"] = len(data["behavior"]["summary"]["file_deleted"])
                    analysis_dictionary["summary"]["file_deleted"]["uniqueTotal"] = len(list(set(data["behavior"]["summary"]["file_deleted"])))
                if "file_created" in data["behavior"]["summary"]:
                    analysis_dictionary["summary"]["file_created"] = {}
                    analysis_dictionary["summary"]["file_created"]["data"] = data["behavior"]["summary"]["file_created"]
                    analysis_dictionary["summary"]["file_created"]["total"] = len(data["behavior"]["summary"]["file_created"])
                    analysis_dictionary["summary"]["file_created"]["uniqueTotal"] = len(list(set(data["behavior"]["summary"]["file_created"])))
                if "file_recreated" in data["behavior"]["summary"]:
                    analysis_dictionary["summary"]["file_recreated"] = {}
                    analysis_dictionary["summary"]["file_recreated"]["data"] = data["behavior"]["summary"]["file_recreated"]
                    analysis_dictionary["summary"]["file_recreated"]["total"] = len(data["behavior"]["summary"]["file_recreated"])
                    analysis_dictionary["summary"]["file_recreated"]["uniqueTotal"] = len(list(set(data["behavior"]["summary"]["file_recreated"])))
                if "directory_created" in data["behavior"]["summary"]:
                    analysis_dictionary["summary"]["directory_created"] = {}
                    analysis_dictionary["summary"]["directory_created"]["data"] = data["behavior"]["summary"]["directory_created"]
                    analysis_dictionary["summary"]["directory_created"]["total"] = len(data["behavior"]["summary"]["directory_created"])
                    analysis_dictionary["summary"]["directory_created"]["uniqueTotal"] = len(list(set(data["behavior"]["summary"]["directory_created"])))
                if "file_opened" in data["behavior"]["summary"]:
                    analysis_dictionary["summary"]["file_opened"] = {}
                    analysis_dictionary["summary"]["file_opened"]["data"] = data["behavior"]["summary"]["file_opened"]
                    analysis_dictionary["summary"]["file_opened"]["total"] = len(data["behavior"]["summary"]["file_opened"])
                    analysis_dictionary["summary"]["file_opened"]["uniqueTotal"] = len(list(set(data["behavior"]["summary"]["file_opened"])))
                if "file_copied" in data["behavior"]["summary"]:
                    analysis_dictionary["summary"]["file_copied"] = {}
                    analysis_dictionary["summary"]["file_copied"]["data"] = data["behavior"]["summary"]["file_copied"]
                    analysis_dictionary["summary"]["file_copied"]["total"] = len(data["behavior"]["summary"]["file_copied"])
                    analysis_dictionary["summary"]["file_copied"]["uniqueTotal"] = len(list(set(data["behavior"]["summary"]["file_copied"])))
                if "file_moved" in data["behavior"]["summary"]:
                    analysis_dictionary["summary"]["file_moved"] = {}
                    analysis_dictionary["summary"]["file_moved"]["data"] = data["behavior"]["summary"]["file_moved"]
                    analysis_dictionary["summary"]["file_moved"]["total"] = len(data["behavior"]["summary"]["file_moved"])
                    analysis_dictionary["summary"]["file_moved"]["uniqueTotal"] = len(list(set(data["behavior"]["summary"]["file_moved"])))
                if "file_written" in data["behavior"]["summary"]:
                    analysis_dictionary["summary"]["file_written"] = {}
                    analysis_dictionary["summary"]["file_written"]["data"] = data["behavior"]["summary"]["file_written"]
                    analysis_dictionary["summary"]["file_written"]["total"] = len(data["behavior"]["summary"]["file_written"])
                    analysis_dictionary["summary"]["file_written"]["uniqueTotal"] = len(list(set(data["behavior"]["summary"]["file_written"])))
                if "file_exists" in data["behavior"]["summary"]:
                    analysis_dictionary["summary"]["file_exists"] = {}
                    analysis_dictionary["summary"]["file_exists"]["data"] = data["behavior"]["summary"]["file_exists"]
                    analysis_dictionary["summary"]["file_exists"]["total"] = len(data["behavior"]["summary"]["file_exists"])
                    analysis_dictionary["summary"]["file_exists"]["uniqueTotal"] = len(list(set(data["behavior"]["summary"]["file_exists"])))
                if "file_read" in data["behavior"]["summary"]:
                    analysis_dictionary["summary"]["file_read"] = {}
                    analysis_dictionary["summary"]["file_read"]["data"] = data["behavior"]["summary"]["file_read"]
                    analysis_dictionary["summary"]["file_read"]["total"] = len(data["behavior"]["summary"]["file_read"])
                    analysis_dictionary["summary"]["file_read"]["uniqueTotal"] = len(list(set(data["behavior"]["summary"]["file_read"])))
                if "directory_enumerated" in data["behavior"]["summary"]:
                    analysis_dictionary["summary"]["directory_enumerated"] = {}
                    analysis_dictionary["summary"]["directory_enumerated"]["data"] = data["behavior"]["summary"]["directory_enumerated"]
                    analysis_dictionary["summary"]["directory_enumerated"]["total"] = len(data["behavior"]["summary"]["directory_enumerated"])
                    analysis_dictionary["summary"]["directory_enumerated"]["uniqueTotal"] = len(list(set(data["behavior"]["summary"]["directory_enumerated"])))

if "processes" in data["behavior"]:
                for proc in data["behavior"]["processes"]:
                    if "calls" in proc:
                        for call in proc["calls"]:

if 'time' in call:
                                if "timeOfCall" not in analysis_dictionary['stats']:
                                    analysis_dictionary['stats']['timeOfCall'] = [call['time']]
                                else:
                                    analysis_dictionary['stats']['timeOfCall'].append(call['time'])

if "totalAPICalls" not in analysis_dictionary['stats']:
                                analysis_dictionary['stats']['totalAPICalls'] = 1
                            else:
                                analysis_dictionary['stats']['totalAPICalls'] = analysis_dictionary['stats']['totalAPICalls'] + 1

if "category" in call:
                                if call["category"] == "file":

if "totalFileCategoryAPICalls" not in analysis_dictionary['stats']:
                                        analysis_dictionary['stats']['totalFileCategoryAPICalls'] = 1
                                    else:
                                        analysis_dictionary['stats']['totalFileCategoryAPICalls'] = analysis_dictionary['stats']['totalFileCategoryAPICalls'] + 1

if "api" in call:

if "fileAPICalls" not in analysis_dictionary:
                                            analysis_dictionary['fileAPICalls'] = {}

if call['api'] not in analysis_dictionary['fileAPICalls']:
                                            analysis_dictionary['fileAPICalls'][call["api"]] = 1
                                        else:
                                            analysis_dictionary['fileAPICalls'][call["api"]] = analysis_dictionary['fileAPICalls'][call["api"]] + 1

if 'uniqueFiles' not in analysis_dictionary:
                                analysis_dictionary['uniqueFiles'] = {}

if 'uniqueFilesTotal' not in analysis_dictionary:
                                analysis_dictionary['uniqueFilesTotal'] = 0

if 'uniqueFileExtensions' not in analysis_dictionary:
                                analysis_dictionary['uniqueFileExtensions'] = {}

if 'uniqueFileExtensionsTotal' not in analysis_dictionary:
                                analysis_dictionary['uniqueFileExtensionsTotal'] = 0

if 'uniqueFileLocations' not in analysis_dictionary:
                                analysis_dictionary['uniqueFileLocations'] = {}

if 'uniqueFileLocationsTotal' not in analysis_dictionary:
                                analysis_dictionary['uniqueFileLocationsTotal'] = 0

if 'arguments' in call:
                                if 'filepath' in call['arguments']:

filePath = call['arguments']['filepath']
                                    file = filePath.split('\\')[-1]
                                    filePathExludingFileName = filePath.rsplit('\\', 1)[0]
                                    fileSplit = file.split('.')
                                    if len(fileSplit) >= 2:
                                        fileName = ''.join(fileSplit[:-1]) 
                                        fileExtension = fileSplit[-1]

if fileName not in analysis_dictionary['uniqueFiles']:
                                            analysis_dictionary['uniqueFiles'][fileName] = 1
                                            analysis_dictionary['uniqueFilesTotal'] = analysis_dictionary['uniqueFilesTotal'] + 1
                                        else:
                                            analysis_dictionary['uniqueFiles'][fileName] = analysis_dictionary['uniqueFiles'][fileName] +1
                                            analysis_dictionary['uniqueFilesTotal'] = analysis_dictionary['uniqueFilesTotal'] + 1

if fileExtension not in analysis_dictionary['uniqueFileExtensions']:
                                            analysis_dictionary['uniqueFileExtensions'][fileExtension] = 1
                                            analysis_dictionary['uniqueFileExtensionsTotal'] = analysis_dictionary['uniqueFileExtensionsTotal'] + 1
                                        else:
                                            analysis_dictionary['uniqueFileExtensions'][fileExtension] = analysis_dictionary['uniqueFileExtensions'][fileExtension] +1
                                            analysis_dictionary['uniqueFileExtensionsTotal'] = analysis_dictionary['uniqueFileExtensionsTotal'] + 1

if filePathExludingFileName not in analysis_dictionary['uniqueFileLocations']:
                                            analysis_dictionary['uniqueFileLocations'][filePathExludingFileName] = 1
                                            analysis_dictionary['uniqueFileLocationsTotal'] = analysis_dictionary['uniqueFileLocationsTotal'] + 1
                                        else:
                                            analysis_dictionary['uniqueFileLocations'][filePathExludingFileName] = analysis_dictionary['uniqueFileLocations'][filePathExludingFileName] +1
                                            analysis_dictionary['uniqueFileLocationsTotal'] = analysis_dictionary['uniqueFileLocationsTotal'] + 1

# ***********************************************************************************

# ------------------------------------------------------------------------------------------------------------------------------------------------------------
# ------------------------------------------------------------------------------------------------------------------------------------------------------------

totalFileCategoryAPICalls = 0 if "totalFileCategoryAPICalls" not in analysis_dictionary['stats'] else analysis_dictionary['stats']['totalFileCategoryAPICalls']

knownExtensions = {
            "documents": [".123",".602",".abw",".accdb",".doc",".docm",".docx",".dot",".dotm",".dotx",".eps",".fb2",".htm",".html",".lrf",".mobi",".odc",".odf",".odg",".odi",".odm",".odp",".ods",".odt",".otg",".oth",".otp",".ots",".ott",".pdb",".pdf",".pot",".potm",".potx",".pps",".ppsx",".ppt",".pptm",".pptx",".ps",".pub",".qpw",".rtf",".sdc",".sdd",".sdw",".sgml",".sla",".slk",".stw",".sxg",".sxi",".sxm",".sxw",".txt",".uop",".uot",".uof",".wdb",".wks",".wpd",".wps",".xhtml",".xml",".xps",".xwp",".csv",".tsv",".ods",".xls",".xlsm",".xlsx",".xlt",".xltm",".xltx",".ods",".numbers",".odg",".pub",".md",".epub",".key",".odt",".sxi",".tex",".wpd",".pages",".txt",".rtf"],
            "images": [".ai",".bmp",".cdr",".cmx",".djvu",".eps",".gif",".ico",".jpeg",".jpg",".png",".ps",".psd",".svg",".tif",".tiff",".wmf",".xbm",".xpm",".webp",".tga",".dds",".j2k",".jfif",".jif",".jpe",".jfif-tbnl",".jpeg-tbnl",".jpg-tbnl",".jpe-tbnl",".jpg-large",".png-large",".webp-large",".gif-large",".jpeg-large",".jpg-large",".raw"],
            "videos": [".3g2",".3gp",".amv",".asf",".avi",".drc",".flv",".flv",".flv",".flv",".flv",".gifv",".m2v",".m4p",".m4v",".mkv",".mkv",".mng",".mov",".mp2",".mp4",".mpe",".mpeg",".mpg",".mpv",".mvv",".ogv",".qt",".rm",".rmvb",".roq",".srt",".svi",".swf",".vob",".webm",".wmv",".yuv"],
            "audio": [".3ga",".aac",".ac3",".aif",".aiff",".amr",".ape",".au",".awb",".dct",".dss",".dvf",".flac",".gsm",".iklax",".ivs",".m4a",".m4b",".m4p",".m4r",".mmf",".mp3",".mpc",".msv",".nmf",".nsf",".ogg",".oga",".mogg",".opus",".ra",".rm",".rmvb",".sln",".tta",".vox",".wav",".wma",".wv",".webm"],
            "databases": [".accdb",".db",".db3",".dbf",".fdb",".gdb",".ldf",".mdf",".mdb",".mde",".myd",".nsf",".odb",".pouch",".pdb",".sdb",".sql",".sqlite",".sqlite3",".xdb",".ydb"],
            "archives": [".001",".7z",".ace",".alz",".arc",".arj",".bz",".bz2",".cab",".cfs",".chm",".cpio",".cpt",".csh",".dar",".dd",".dgc",".dmg",".ear",".gz",".hqx",".ice",".jar",".kgb",".lbr",".lha",".lzh",".lzx",".mhtml",".mie",".pak",".paq6",".paq7",".paq8",".par",".par2",".pea",".pim",".pit",".qda",".rar",".rk",".sda",".sea",".sen",".sfark",".sfx",".shar",".sit",".sitx",".sqx",".tar",".tbz2",".tgz",".tlz",".tz",".uha",".uue",".war",".wim",".xar",".xp3",".xz",".yz1",".z",".zip",".zipx",".zoo",".zpaq",".zz"],
            "executable": [".apk",".app",".bat",".bin",".cmd",".com",".cpl",".dll",".exe",".gadget",".hta",".inf",".ins",".inx",".ipa",".isu",".jar",".js",".jse",".lnk",".msc",".msi",".msp",".mst",".osx",".out",".paf",".pif",".prc",".ps1",".reg",".rgs",".run",".scr",".sct",".shb",".shs",".u3p",".vb",".vbe",".vbs",".vbscript",".ws",".wsf",".wsh"],
            "system": [".386",".admx",".appx",".bak",".bak1",".bak2",".bak3",".bak4",".bat",".bin",".cab",".cat",".cfg",".cpl",".cur",".dll",".dmp",".drv",".grp",".icns",".ico",".ini",".job",".key",".lnk",".log",".man",".mgr",".msc",".msi",".msp",".mui",".nfo",".ocx",".pdb",".pif",".pl",".pm",".pol",".pps",".prf",".ps1",".reg",".rgs",".scr",".sys",".tff",".ttf",".wpl"],
            "backup": [".bak",".backup",".bu",".old",".orig",".temp",".tmp",".swp",".swo",".swn",".swo",".1st",".0",".1",".2",".3",".4",".5",".6",".7",".8",".9",".001",".002",".003",".004",".005",".006",".007",".008",".009",".0010",".0011",".0012",".0013",".0014",".0015",".0016",".0017",".0018",".0019",".0020",".0021",".0022",".0023",".0024",".0025",".0026",".0027",".0028",".0029",".0030",".0031",".0032",".0033",".0034",".0035",".0036",".0037",".0038",".0039",".0040",".0041",".0042",".0043",".0044",".0045",".0046",".0047",".0048",".0049",".0050",".0051",".0052",".0053",".0054",".0055",".0056",".0057",".0058",".0059",".0060",".0061",".0062",".0063",".0064",".0065",".0066",".0067",".0068",".0069",".0070",".0071",".0072",".0073",".0074",".0075",".0076",".0077",".0078",".0079",".0080",".0081",".0082",".0083",".0084",".0085",".0086",".0087",".0088",".0089",".0090",".0091",".0092",".0093",".0094",".0095",".0096",".0097",".0098",".0099",".01",".02",".03",".04",".05",".06",".07",".08",".09",".001",".002",".003",".004",".005",".006",".007",".008",".009"],
            "virtualMachine": [".vmx",".vbox",".vdi",".vhd",".qcow2",".ova",".vmsd",".vmtm",".vswp",".nvram"],
            "email": [".dbx",".eml",".emlx",".mbox",".msg",".pst",".ost",".mbx",".tbb",".mht",".nws",".tnef"],
            "games": [".sav",".cfg",".ini",".pak",".wad",".gam",".nes",".snes",".gen",".gba",".n64",".d64",".rom",".iso",".bin",".cue",".cso",".chd",".gdi",".cdi",".elf",".prx",".nds",".3ds",".chm",".dat",".rez",".map",".m3u",".xm",".mod",".s3m",".it",".adlib",".mus",".xma",".mpq",".bb",".ddraw",".voi",".aoe",".bar",".mpk",".w3g",".dol",".bgl",".ttf",".bik",".bik2",".pss",".blk",".bmd",".blp",".brres",".dem",".dem2",".dem3",".dem4",".dmo",".elf",".epk",".ezs",".fsb",".gbs",".gsb",".gsc",".gsm",".h4m",".ilm",".kra",".l3d",".l3p",".l3t",".lpk",".lst",".m3p",".m3s",".m4a",".m4b",".m4s",".m4v",".mgz",".mmp",".mps",".mpp",".nmp",".nut",".omod",".osu",".osz",".p8",".p8.png",".pk3",".pk4",".pk5",".pk6",".pk7",".pk8",".pk9",".pke",".pkg",".pkh",".pkk",".pkm",".pko",".pkp",".pkr",".pks",".pkt",".pku",".pkv",".pkw",".pkx",".pky",".pkz",".plr",".plz",".rez",".res",".rgd",".rkv",".rom",".rpz",".rrk",".rtb",".rvdata2",".rwdata",".rxdata",".sbk",".sbl",".sdb",".sdg",".sdl",".sdw",".sep",".sgc",".sgh",".shc",".slc",".snd",".snd0",".sndt",".sndx",".snp",".sod",".sof",".ssf",".ssg",".ssf",".ssg",".sso",".sst",".sts",".szt",".tem",".ucl",".udk",".umap",".umx",".unr",".unx",".unx",".uop",".usa",".usm",".utr",".v64",".vl2",".vpk",".w3x",".wad",".wdl",".wfav",".wgp",".wl6",".wlk",".wmo",".x2m",".xma",".xp3",".xxx",".ydk",".ydr",".ypt",".z64"],
            "development": [".c",".cpp",".h",".hpp",".java",".py",".cs",".html",".css",".js",".php",".rb",".swift",".go",".perl",".sql",".json",".xml",".yaml",".ini",".cfg",".md",".txt",".log",".gitignore",".dockerignore",".bat",".sh",".ps1",".makefile",".yml",".conf",".properties",".gradle",".classpath",".project",".sln",".vcxproj",".suo",".dll",".lib",".obj",".o",".a",".so",".exe",".app",".ipa",".apk",".jar",".war",".ear",".class",".dmg",".pkg",".deb",".rpm",".tar.gz",".zip",".7z",".rar",".bz2",".gz",".tar",".tar.bz2",".tar.xz",".img",".iso",".vhd",".vhdx",".vdi",".vmdk",".bak",".swp",".swo",".swn",".swo",".old",".bak",".orig",".backup",".temp",".tmp",".cache",".test",".bak1",".bak2",".bak3",".bu"]
        }

analysis_dictionary["fileLocationCounter"] = {
            "Documents": 0,
            "Pictures": 0,
            "Music": 0,
            "Videos": 0,
            "Desktop": 0,
            "Downloads": 0,
            "Other": 0
        }

if "uniqueFileLocations" in analysis_dictionary:
            for val in analysis_dictionary['uniqueFileLocations']:
                if "Documents" in val:
                    analysis_dictionary["fileLocationCounter"]["Documents"] = analysis_dictionary["fileLocationCounter"]["Documents"] + analysis_dictionary['uniqueFileLocations'][val]
                elif "Pictures" in val:
                    analysis_dictionary["fileLocationCounter"]["Pictures"] = analysis_dictionary["fileLocationCounter"]["Pictures"] + analysis_dictionary['uniqueFileLocations'][val]
                elif "Music" in val:
                    analysis_dictionary["fileLocationCounter"]["Music"] = analysis_dictionary["fileLocationCounter"]["Music"] + analysis_dictionary['uniqueFileLocations'][val]
                elif "Videos" in val:
                    analysis_dictionary["fileLocationCounter"]["Videos"] = analysis_dictionary["fileLocationCounter"]["Videos"] + analysis_dictionary['uniqueFileLocations'][val]
                elif "Desktop" in val:
                    analysis_dictionary["fileLocationCounter"]["Desktop"] = analysis_dictionary["fileLocationCounter"]["Desktop"] + analysis_dictionary['uniqueFileLocations'][val]
                elif "Downloads" in val:
                    analysis_dictionary["fileLocationCounter"]["Downloads"] = analysis_dictionary["fileLocationCounter"]["Downloads"] + analysis_dictionary['uniqueFileLocations'][val]
                else:
                    analysis_dictionary["fileLocationCounter"]["Other"] = analysis_dictionary["fileLocationCounter"]["Other"] + analysis_dictionary['uniqueFileLocations'][val]

timeDiff = 0
        if "timeOfCall" in analysis_dictionary["stats"]:
            analysis_dictionary["stats"]["timeOfCall"].sort()
            timeDiff = analysis_dictionary["stats"]["timeOfCall"][-1] - analysis_dictionary["stats"]["timeOfCall"][1]

analysis_dictionary["fileExtensionsCounter"] = {
            "documents": 0,
            "images": 0,
            "videos": 0,
            "audio": 0,
            "databases": 0,
            "archives": 0,
            "executable": 0,
            "system": 0,
            "backup": 0,
            "virtualMachine": 0,
            "email": 0,
            "games": 0,
            "development": 0,
            "unknown": 0
        }

if "uniqueFileExtensions" in analysis_dictionary:
            for val in analysis_dictionary['uniqueFileExtensions']:
                if "."+val in knownExtensions['documents']:
                    analysis_dictionary["fileExtensionsCounter"]["documents"] = analysis_dictionary["fileExtensionsCounter"]["documents"] + analysis_dictionary['uniqueFileExtensions'][val]
                elif "."+val in knownExtensions['images']:
                    analysis_dictionary["fileExtensionsCounter"]["images"] = analysis_dictionary["fileExtensionsCounter"]["images"] + analysis_dictionary['uniqueFileExtensions'][val]
                elif "."+val in knownExtensions['videos']:
                    analysis_dictionary["fileExtensionsCounter"]["videos"] = analysis_dictionary["fileExtensionsCounter"]["videos"] + analysis_dictionary['uniqueFileExtensions'][val]
                elif "."+val in knownExtensions['audio']:
                    analysis_dictionary["fileExtensionsCounter"]["audio"] = analysis_dictionary["fileExtensionsCounter"]["audio"] + analysis_dictionary['uniqueFileExtensions'][val]
                elif "."+val in knownExtensions['databases']:
                    analysis_dictionary["fileExtensionsCounter"]["databases"] = analysis_dictionary["fileExtensionsCounter"]["databases"] + analysis_dictionary['uniqueFileExtensions'][val]
                elif "."+val in knownExtensions['archives']:
                    analysis_dictionary["fileExtensionsCounter"]["archives"] = analysis_dictionary["fileExtensionsCounter"]["archives"] + analysis_dictionary['uniqueFileExtensions'][val]
                elif "."+val in knownExtensions['executable']:
                    analysis_dictionary["fileExtensionsCounter"]["executable"] = analysis_dictionary["fileExtensionsCounter"]["executable"] + analysis_dictionary['uniqueFileExtensions'][val]
                elif "."+val in knownExtensions['system']:
                    analysis_dictionary["fileExtensionsCounter"]["system"] = analysis_dictionary["fileExtensionsCounter"]["system"] + analysis_dictionary['uniqueFileExtensions'][val]
                elif "."+val in knownExtensions['backup']:
                    analysis_dictionary["fileExtensionsCounter"]["backup"] = analysis_dictionary["fileExtensionsCounter"]["backup"] + analysis_dictionary['uniqueFileExtensions'][val]
                elif "."+val in knownExtensions['virtualMachine']:
                    analysis_dictionary["fileExtensionsCounter"]["virtualMachine"] = analysis_dictionary["fileExtensionsCounter"]["virtualMachine"] + analysis_dictionary['uniqueFileExtensions'][val]
                elif "."+val in knownExtensions['email']:
                    analysis_dictionary["fileExtensionsCounter"]["email"] = analysis_dictionary["fileExtensionsCounter"]["email"] + analysis_dictionary['uniqueFileExtensions'][val]
                elif "."+val in knownExtensions['games']:
                    analysis_dictionary["fileExtensionsCounter"]["games"] = analysis_dictionary["fileExtensionsCounter"]["games"] + analysis_dictionary['uniqueFileExtensions'][val]
                elif "."+val in knownExtensions['development']:
                    analysis_dictionary["fileExtensionsCounter"]["development"] = analysis_dictionary["fileExtensionsCounter"]["development"] + analysis_dictionary['uniqueFileExtensions'][val]
                else:
                    analysis_dictionary["fileExtensionsCounter"]["unknown"] = analysis_dictionary["fileExtensionsCounter"]["unknown"] + analysis_dictionary['uniqueFileExtensions'][val]

#************************************************************************************************************************************************************
#************************************************************************************************************************************************************

#^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
#^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

# default values

F1_totalAPICalls = 0
        F2_percentageOfFileAPICalls = 0
        F3_1_fileAPIPercentageNtReadFile = 0
        F3_2_fileAPIPercentageNtWriteFile = 0
        F3_3_fileAPIPercentageSetFilePointer = 0
        F3_4_fileAPIPercentageSetFilePointerEx = 0
        F3_5_fileAPIPercentageNtCreateFile = 0
        F3_6_fileAPIPercentageNtDeviceIoControlFile = 0
        F3_7_fileAPIPercentageGetFileAttributesExW = 0
        F3_8_fileAPIPercentageNtQueryDirectoryFile = 0
        F3_9_fileAPIPercentageFindFirstFileExW = 0
        F3_10_fileAPIPercentageNtQueryInformationFile = 0
        F3_11_fileAPIPercentageGetFileSizeEx = 0
        F3_12_fileAPIPercentageGetFileType = 0
        F3_13_fileAPIPercentageNtOpenFile = 0
        F3_14_fileAPIPercentageSetEndOfFile = 0
        F3_15_fileAPIPercentageMoveFileWithProgressW = 0
        F3_16_fileAPIPercentageSetFileAttributesW = 0
        F3_17_fileAPIPercentageDeleteFileW = 0
        F4_percentageOfUniqueFilesTouched = 0
        F5_percentageOfUniqueFileExtensionsTouched = 0
        F6_percentageOfUniqueFileLocationsTouched = 0
        F7_1_percentgeOfFileExtensionPy = 0
        F7_2_percentgeOfFileExtensionPng = 0
        F7_3_percentgeOfFileExtensionTxt = 0
        F7_4_percentgeOfFileExtensionTlc = 0
        F7_5_percentgeOfFileExtensionMsg = 0
        F7_6_percentgeOfFileExtensionExe = 0
        F7_7_percentageOfFileExtensionDll = 0
        F8_1_percentageOfFilesInDocuments = 0
        F8_2_percentageOfFilesInPictures = 0
        F8_3_percentageOfFilesInMusic = 0
        F8_4_percentageOfFilesInVideos = 0
        F8_5_percentageOfFilesInDesktop = 0
        F8_6_percentageOfFilesInDownloads = 0
        F9_APICallsPerSecond = 0
        F10_percentageOfPotentialCustomExtensionsUsed = 0
        F11_1_percentageOfFileExtensionsDocuments = 0
        F11_2_percentageOfFileExtensionsImages = 0
        F11_3_percentageOfFileExtensionsVideos = 0
        F11_4_percentageOfFileExtensionsAudio = 0
        F11_5_percentageOfFileExtensionsDatabase = 0
        F11_6_percentageOfFileExtensionsArchives = 0
        F11_7_percentageOfFileExtensionsExecutables = 0
        F11_8_percentageOfFileExtensionsSystem = 0
        F11_9_percentageOfFileExtensionsBackup = 0
        F11_10_percentageOfFileExtensionsEmail = 0
        F11_11_percentageOfFileExtensionsVMImages = 0
        F11_12_percentageOfFileExtensionsGames = 0
        F11_13_percentageOfFileExtensionsDevelopment = 0

features_labels = ["sha256", "label",
                           "PercentageOfFileAPICalls",
                           "PercentageOfUniqueFiles",
                           "PercentageOfUniqueFileLocations",
                           "PercentageOfUniqueFileExtensions",
                           "PercentageOfPotentialCustomExtensions",
                           "PercentageOfRansomwareExtensions",
                           "PercentageOfFileExtensionsDocuments", "PercentageOfFileExtensionsImages", "PercentageOfFileExtensionsVideos", "PercentageOfFileExtensionsAudio", "PercentageOfFileExtensionsDatabase","PercentageOfFileExtensionsArchives","PercentageOfFileExtensionsExecutables","PercentageOfFileExtensionsSystem", "PercentageOfFileExtensionsBackup","PercentageOfFileExtensionsEmail", "PercentageOfFileExtensionsVMImages","PercentageOfFileExtensionsGames","PercentageOfFileExtensionsDevelopment",
                           "APICallsPerSecond"
                           ]
        features = []

F1_totalAPICalls = 0 if "totalAPICalls" not in analysis_dictionary['stats'] else analysis_dictionary['stats']['totalAPICalls']

F2_percentageOfFileAPICalls = 0 if F1_totalAPICalls == 0 else round((totalFileCategoryAPICalls / F1_totalAPICalls) * 100)

if "fileAPICalls" in analysis_dictionary and totalFileCategoryAPICalls != 0:
            F3_1_fileAPIPercentageNtReadFile = 0 if "NtReadFile" not in analysis_dictionary['fileAPICalls'] else round((analysis_dictionary['fileAPICalls']['NtReadFile'] / totalFileCategoryAPICalls) * 100)
            F3_2_fileAPIPercentageNtWriteFile = 0 if "NtWriteFile" not in analysis_dictionary['fileAPICalls'] else round((analysis_dictionary['fileAPICalls']['NtWriteFile'] / totalFileCategoryAPICalls) * 100)
            F3_3_fileAPIPercentageSetFilePointer = 0 if "SetFilePointer" not in analysis_dictionary['fileAPICalls'] else round((analysis_dictionary['fileAPICalls']['SetFilePointer'] / totalFileCategoryAPICalls) * 100)
            F3_4_fileAPIPercentageSetFilePointerEx = 0 if "SetFilePointerEx" not in analysis_dictionary['fileAPICalls'] else round((analysis_dictionary['fileAPICalls']['SetFilePointerEx'] / totalFileCategoryAPICalls) * 100)
            F3_5_fileAPIPercentageNtCreateFile = 0 if "NtCreateFile" not in analysis_dictionary['fileAPICalls'] else round((analysis_dictionary['fileAPICalls']['NtCreateFile'] / totalFileCategoryAPICalls) * 100)
            F3_6_fileAPIPercentageNtDeviceIoControlFile= 0 if "NtDeviceIoControlFile" not in analysis_dictionary['fileAPICalls'] else round((analysis_dictionary['fileAPICalls']['NtDeviceIoControlFile'] / totalFileCategoryAPICalls) * 100)
            F3_7_fileAPIPercentageGetFileAttributesExW = 0 if "GetFileAttributesExW" not in analysis_dictionary['fileAPICalls'] else round((analysis_dictionary['fileAPICalls']['GetFileAttributesExW'] / totalFileCategoryAPICalls) * 100)
            F3_8_fileAPIPercentageNtQueryDirectoryFile = 0 if "NtQueryDirectoryFile" not in analysis_dictionary['fileAPICalls'] else round((analysis_dictionary['fileAPICalls']['NtQueryDirectoryFile'] / totalFileCategoryAPICalls) * 100)
            F3_9_fileAPIPercentageFindFirstFileExW = 0 if "FindFirstFileExW" not in analysis_dictionary['fileAPICalls'] else round((analysis_dictionary['fileAPICalls']['FindFirstFileExW'] / totalFileCategoryAPICalls) * 100)
            F3_10_fileAPIPercentageNtQueryInformationFile = 0 if "NtQueryInformationFile" not in analysis_dictionary['fileAPICalls'] else round((analysis_dictionary['fileAPICalls']['NtQueryInformationFile'] / totalFileCategoryAPICalls) * 100)
            F3_11_fileAPIPercentageGetFileSizeEx = 0 if "GetFileSizeEx" not in analysis_dictionary['fileAPICalls'] else round((analysis_dictionary['fileAPICalls']['GetFileSizeEx'] / totalFileCategoryAPICalls) * 100)
            F3_12_fileAPIPercentageGetFileType = 0 if "GetFileType" not in analysis_dictionary['fileAPICalls'] else round((analysis_dictionary['fileAPICalls']['GetFileType'] / totalFileCategoryAPICalls) * 100)
            F3_13_fileAPIPercentageNtOpenFile = 0 if "NtOpenFile" not in analysis_dictionary['fileAPICalls'] else round((analysis_dictionary['fileAPICalls']['NtOpenFile'] / totalFileCategoryAPICalls) * 100)
            F3_14_fileAPIPercentageSetEndOfFile = 0 if "SetEndOfFile" not in analysis_dictionary['fileAPICalls'] else round((analysis_dictionary['fileAPICalls']['SetEndOfFile'] / totalFileCategoryAPICalls) * 100)
            F3_15_fileAPIPercentageMoveFileWithProgressW = 0 if "MoveFileWithProgressW" not in analysis_dictionary['fileAPICalls'] else round((analysis_dictionary['fileAPICalls']['MoveFileWithProgressW'] / totalFileCategoryAPICalls) * 100)
            F3_16_fileAPIPercentageSetFileAttributesW = 0 if "SetFileAttributesW" not in analysis_dictionary['fileAPICalls'] else round((analysis_dictionary['fileAPICalls']['SetFileAttributesW'] / totalFileCategoryAPICalls) * 100)
            F3_17_fileAPIPercentageDeleteFileW = 0 if "DeleteFileW" not in analysis_dictionary['fileAPICalls'] else round((analysis_dictionary['fileAPICalls']['DeleteFileW'] / totalFileCategoryAPICalls) * 100)

if "uniqueFiles" in analysis_dictionary and "uniqueFilesTotal" in analysis_dictionary and analysis_dictionary['uniqueFilesTotal'] != 0:
            F4_percentageOfUniqueFilesTouched = round((len(analysis_dictionary['uniqueFiles']) / analysis_dictionary['uniqueFilesTotal']) * 100)
        if "uniqueFileExtensions" in analysis_dictionary and "uniqueFileExtensionsTotal" in analysis_dictionary and analysis_dictionary['uniqueFileExtensionsTotal'] != 0:
            F5_percentageOfUniqueFileExtensionsTouched = round((len(analysis_dictionary['uniqueFileExtensions']) / analysis_dictionary['uniqueFileExtensionsTotal']) * 100)
        if "uniqueFiles" in analysis_dictionary and "uniqueFilesTotal" in analysis_dictionary and analysis_dictionary['uniqueFilesTotal'] != 0:
            F6_percentageOfUniqueFileLocationsTouched = round((len(analysis_dictionary['uniqueFiles']) / analysis_dictionary['uniqueFilesTotal']) * 100)

if "uniqueFileExtensions" in analysis_dictionary and "uniqueFileExtensionsTotal" in analysis_dictionary and analysis_dictionary['uniqueFileExtensionsTotal'] != 0:
            F7_1_percentgeOfFileExtensionPy = 0 if 'py' not in analysis_dictionary['uniqueFileExtensions'] else round((analysis_dictionary['uniqueFileExtensions']['py'] / analysis_dictionary['uniqueFileExtensionsTotal']) * 100)
            F7_2_percentgeOfFileExtensionPng = 0 if 'png' not in analysis_dictionary['uniqueFileExtensions'] else round((analysis_dictionary['uniqueFileExtensions']['png'] / analysis_dictionary['uniqueFileExtensionsTotal']) * 100)
            F7_3_percentgeOfFileExtensionTxt = 0 if 'txt' not in analysis_dictionary['uniqueFileExtensions'] else round((analysis_dictionary['uniqueFileExtensions']['txt'] / analysis_dictionary['uniqueFileExtensionsTotal']) * 100)
            F7_4_percentgeOfFileExtensionTlc = 0 if 'tlc' not in analysis_dictionary['uniqueFileExtensions'] else round((analysis_dictionary['uniqueFileExtensions']['tlc'] / analysis_dictionary['uniqueFileExtensionsTotal']) * 100)
            F7_5_percentgeOfFileExtensionMsg = 0 if 'msg' not in analysis_dictionary['uniqueFileExtensions'] else round((analysis_dictionary['uniqueFileExtensions']['msg'] / analysis_dictionary['uniqueFileExtensionsTotal']) * 100)
            F7_6_percentgeOfFileExtensionExe = 0 if 'exe' not in analysis_dictionary['uniqueFileExtensions'] else round((analysis_dictionary['uniqueFileExtensions']['exe'] / analysis_dictionary['uniqueFileExtensionsTotal']) * 100)
            F7_7_percentageOfFileExtensionDll = 0 if 'dll' not in analysis_dictionary['uniqueFileExtensions'] else round((analysis_dictionary['uniqueFileExtensions']['dll'] / analysis_dictionary['uniqueFileExtensionsTotal']) * 100)

if "fileLocationCounter" in analysis_dictionary and "uniqueFileLocationsTotal" in analysis_dictionary and analysis_dictionary['uniqueFileLocationsTotal'] != 0:
            F8_1_percentageOfFilesInDocuments = round((analysis_dictionary["fileLocationCounter"]["Documents"] / analysis_dictionary['uniqueFileLocationsTotal']) * 100)
            F8_2_percentageOfFilesInPictures = round((analysis_dictionary["fileLocationCounter"]["Pictures"] / analysis_dictionary['uniqueFileLocationsTotal']) * 100)
            F8_3_percentageOfFilesInMusic = round((analysis_dictionary["fileLocationCounter"]["Music"] / analysis_dictionary['uniqueFileLocationsTotal']) * 100)
            F8_4_percentageOfFilesInVideos = round((analysis_dictionary["fileLocationCounter"]["Videos"] / analysis_dictionary['uniqueFileLocationsTotal']) * 100)
            F8_5_percentageOfFilesInDesktop = round((analysis_dictionary["fileLocationCounter"]["Desktop"] / analysis_dictionary['uniqueFileLocationsTotal']) * 100)
            F8_6_percentageOfFilesInDownloads = round((analysis_dictionary["fileLocationCounter"]["Downloads"] / analysis_dictionary['uniqueFileLocationsTotal']) * 100)

if "stats" in analysis_dictionary and "timeOfCall" in analysis_dictionary["stats"] and timeDiff != 0:
            F9_APICallsPerSecond = round((len(analysis_dictionary["stats"]["timeOfCall"]) / timeDiff))

if "fileExtensionsCounter" in analysis_dictionary and "uniqueFileExtensionsTotal" in analysis_dictionary and analysis_dictionary['uniqueFileExtensionsTotal'] != 0:
            F11_1_percentageOfFileExtensionsDocuments = round((analysis_dictionary["fileExtensionsCounter"]["documents"] / analysis_dictionary['uniqueFileExtensionsTotal']) * 100)
            F11_2_percentageOfFileExtensionsImages = round((analysis_dictionary["fileExtensionsCounter"]["images"] / analysis_dictionary['uniqueFileExtensionsTotal']) * 100)
            F11_3_percentageOfFileExtensionsVideos = round((analysis_dictionary["fileExtensionsCounter"]["videos"] / analysis_dictionary['uniqueFileExtensionsTotal']) * 100)
            F11_4_percentageOfFileExtensionsAudio = round((analysis_dictionary["fileExtensionsCounter"]["audio"] / analysis_dictionary['uniqueFileExtensionsTotal']) * 100)
            F11_5_percentageOfFileExtensionsDatabase = round((analysis_dictionary["fileExtensionsCounter"]["databases"] / analysis_dictionary['uniqueFileExtensionsTotal']) * 100)
            F11_6_percentageOfFileExtensionsArchives = round((analysis_dictionary["fileExtensionsCounter"]["archives"] / analysis_dictionary['uniqueFileExtensionsTotal']) * 100)
            F11_7_percentageOfFileExtensionsExecutables = round((analysis_dictionary["fileExtensionsCounter"]["executable"] / analysis_dictionary['uniqueFileExtensionsTotal']) * 100)
            F11_8_percentageOfFileExtensionsSystem = round((analysis_dictionary["fileExtensionsCounter"]["system"] / analysis_dictionary['uniqueFileExtensionsTotal']) * 100)
            F11_9_percentageOfFileExtensionsBackup = round((analysis_dictionary["fileExtensionsCounter"]["backup"] / analysis_dictionary['uniqueFileExtensionsTotal']) * 100)
            F11_10_percentageOfFileExtensionsEmail = round((analysis_dictionary["fileExtensionsCounter"]["virtualMachine"] / analysis_dictionary['uniqueFileExtensionsTotal']) * 100)
            F11_11_percentageOfFileExtensionsVMImages = round((analysis_dictionary["fileExtensionsCounter"]["email"] / analysis_dictionary['uniqueFileExtensionsTotal']) * 100)
            F11_12_percentageOfFileExtensionsGames = round((analysis_dictionary["fileExtensionsCounter"]["games"] / analysis_dictionary['uniqueFileExtensionsTotal']) * 100)
            F11_13_percentageOfFileExtensionsDevelopment = round((analysis_dictionary["fileExtensionsCounter"]["development"] / analysis_dictionary['uniqueFileExtensionsTotal']) * 100)

RansomwareExtensions = round((F7_1_percentgeOfFileExtensionPy + F7_2_percentgeOfFileExtensionPng + F7_3_percentgeOfFileExtensionTxt + F7_4_percentgeOfFileExtensionTlc + F7_5_percentgeOfFileExtensionMsg + F7_6_percentgeOfFileExtensionExe + F7_7_percentageOfFileExtensionDll ) / 7)
        label = "M" if malicious else "B"
        features.append([sha256, label,
                         F2_percentageOfFileAPICalls,
                         F4_percentageOfUniqueFilesTouched,
                         F6_percentageOfUniqueFileLocationsTouched,
                         F5_percentageOfUniqueFileExtensionsTouched,
                         F10_percentageOfPotentialCustomExtensionsUsed,
                         RansomwareExtensions,
                         F11_1_percentageOfFileExtensionsDocuments, F11_2_percentageOfFileExtensionsImages, F11_3_percentageOfFileExtensionsVideos, F11_4_percentageOfFileExtensionsAudio, F11_5_percentageOfFileExtensionsDatabase, F11_6_percentageOfFileExtensionsArchives, F11_7_percentageOfFileExtensionsExecutables, F11_8_percentageOfFileExtensionsSystem, F11_9_percentageOfFileExtensionsBackup, F11_10_percentageOfFileExtensionsEmail, F11_11_percentageOfFileExtensionsVMImages, F11_12_percentageOfFileExtensionsGames, F11_13_percentageOfFileExtensionsDevelopment,
                         F9_APICallsPerSecond
                         ]
                        )             
        
        ###
        return features_labels, features, sha256
        
    except Exception as e:
        log("|process error| " + filename + " -> " + sha256 + " (" + str(e) + ")")
        pass

July 10, 2024, 3:29 p.m. Public Create New Vesion Download Dataset (472.7 KB)

Please wait ...

Public Datasets

File Operations for Ransomware Detection Rows: 2652 | Cols: 23 | Reports Used: 3084

Reports Used

Dataset Code

Comments

No comments to display